{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Functional API implementation of a three-output model (7.3)\n",
    "a network that takes as input a series\n",
    "of social media posts from a single anonymous person and tries to predict attributes of\n",
    "that person, such as age, gender, and income level "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras import layers, Input\n",
    "from keras.models import Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "vocabulary_size = 50000\n",
    "num_income_groups = 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "posts_input = Input(shape=(None,), dtype='int32', name='posts')\n",
    "embedded_posts = layers.Embedding(256, vocabulary_size)(posts_input)\n",
    "x = layers.Conv1D(128, 5, activation='relu')(embedded_posts)\n",
    "x = layers.MaxPooling1D(5)(x)\n",
    "x = layers.Conv1D(256, 5, activation='relu')(x)\n",
    "x = layers.Conv1D(256, 5, activation='relu')(x)\n",
    "x = layers.MaxPooling1D(5)(x)\n",
    "x = layers.Conv1D(256, 5, activation='relu')(x)\n",
    "x = layers.Conv1D(256, 5, activation='relu')(x)\n",
    "x = layers.GlobalMaxPooling1D()(x)\n",
    "x = layers.Dense(128, activation='relu')(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# the above features are now pass put to not just one but all three\n",
    "# Note that the output layers are given names \n",
    "\n",
    "age_prediction = layers.Dense(1, name='age')(x)\n",
    "\n",
    "income_prediction = layers.Dense(num_income_groups,activation='softmax',name='income')(x)\n",
    "\n",
    "gender_prediction = layers.Dense(1, activation='sigmoid', name='gender')(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Model(posts_input,[age_prediction, income_prediction, gender_prediction])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " we can pass it as a list or Dictionary\n",
    " \n",
    " The approch below is not good because very imbalanced loss contributions will cause the model representations to\n",
    " be optimized preferentially for the task with the largest individual loss, at the expense of the other tasks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#model.compile(optimizer='rmsprop',loss=['mse', 'categorical_crossentropy', 'binary_crossentropy'])\n",
    "\n",
    "#model.compile(optimizer='rmsprop',loss={'age': 'mse','income': 'categorical_crossentropy','gender': 'binary_crossentropy'})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " the mean squared error (MSE) loss\n",
    "used for the age-regression task typically takes a value around 3–5, whereas the crossentropy loss used for the gender-classification task can be as low as 0.1. In such a situation, to balance the contribution of the different losses, you can assign a weight of 10\n",
    "to the crossentropy loss and a weight of 0.25 to the MSE loss."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Compilation options of a multi-output model: loss weighting "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Multi output layer ke loss function sb ke different honge\n",
    ",But sb ko milake gradient dicent use kerte hai\n",
    ",But sb loss function ke range different honge\n",
    ",to is liay loss ko weights denge take sb ke value kareb ajai"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='rmsprop',\n",
    "            loss=['mse', 'categorical_crossentropy', 'binary_crossentropy'],\n",
    "            loss_weights=[0.25, 1., 10.])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# or pass as dictionary\n",
    "\n",
    "#model.compile(optimizer='rmsprop',\n",
    "#            loss={'age': 'mse','income': 'categorical_crossentropy','gender': 'binary_crossentropy'},\n",
    "#            loss_weights={'age': 0.25,'income': 1.,'gender': 10.})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Feeding data to a multi-output model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.fit(posts, [age_targets, income_targets, gender_targets],epochs=10, batch_size=64)\n",
    "\n",
    "# or as Dictionary\n",
    "#model.fit(posts, {'age': age_targets,'income': income_targets,\n",
    "#        'gender': gender_targets},epochs=10, batch_size=64)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We didn't pass any input because we dont have any, thats why its giving error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
